At work I've been moving forward with upgrading various things that we rely upon: OS, DB, etc.
I created a custom distro of Red Hat Enterprise Linux 5 64-bit to replace the previous custom distro of Red Hat Enterprise Linux 4 Update 3 32-bit. (Custom meaning set of custom kickstart scripts and some extra packages slipstreamed in with the default ones.)
I've been working with one of our contractors to move from Oracle 10.1.0.3 32-bit to Oracle 10.2.0.3 64-bit (and only install the bits and pieces we are actually using).
This week I installed my custom RHEL5 into a VM, updated our installation scripts to use the new version of Oracle, installed Oracle in the VM, installed all of the Perl modules that we use from CPAN and installed our product over top of it all.
I tried to start Apache and it failed. It turns out that I had to tweak the httpd.conf that we ship since we previously used Apache 2.0.52 and RHEL5 ships with Apache 2.2.3. Once I got past that I found that I had permission issues with $ENV{ORACLE_HOME}/lib. I fixed that only to find that I still have problems to overcome.
ERROR OCIEnvNlsCreate. Check ORACLE_HOME env var, NLS settings, permissions, etc.
It's not $ENV{ORACLE_HOME}, $ENV{ORACLE_SID} or the NLS stuff. It's definitely file and/or directory permissions but the problem is that I don't know which files and/or directories... yet.
How do I know it's permissions? I cheated. I did a 'chmod -R a+rx /u01 /u02' and all of a sudden I could use our app just fine.
I'm doing a new install from scratch so that I can play around as a normal user and see what strace tells me when I run various scripts.
I really love strace when I need to find problems like this.
strace -o /tmp/trace.txt /path/to/failing/script vi /tmp/trace.txt